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LEARNING TYPE IMAGE CLASSIFICATION APPARATUS, METHOD 
THEREOF AND PROCESSING RECORDING MEDIUM ON WHICH 
PROCESSING PROGRAM IS RECORDED 

CROSS-REFERENCE TO RELATED APPLICATIONS 
This application is based upon and claims the 
benefit of priority from the prior Japanese Patent 
Application No. 2000-217720, filed July 18, 2000, the 
entire contents of which are incorporated herein by 
reference, 

BACKGROUND OF THE INVENTION 

1. Field of the Invention 

A large quantity of labor is required in the work 
of manually classifying and putting in order an 
infinite number of images present in the database, in 
the computer, or on the network and of attaching key 
words for retrieval with hands. 

2. Description of the Related Art 
Therefore, there are proposed an automatic image 

classification apparatus and an apparatus of 
automatically adding keywords for image retrieval. 

As an example of the automatic image 
classification apparatus, there is available Jpn. Pat. 
Appln. KOKAI Publication No. 11-328422. Furthermore, 
an apparatus of automatically adding key words for 
retrieval, there are available Jpn. Pat. Appln. KOKAI 
Publication No. 8-249349, Jpn. Pat. Appln. KOKAI 



Publication No. 10-49542, and Jpn. Pat. Appln. KOKAI 
Publication No. 10-55366 and the like. 

In the beginning, in Jpn. Pat. Appln. KOKAI 
Publication No. 11-328442, there is proposed an 
apparatus of automatically classifying, for example, 
natural images and artificial images by using features 
extracted from images by sampling. 

Although there is a possibility that this 
apparatus which is capable of classifying images 
functions well depending upon the features distributed 
uniformly over the whole image, but the apparatus is 
not sufficient for classifying objects locally present 
in images. 

That is, it is necessary to devise means for 
clipping only object regions in order to classify 
objects . 

Such devise is introduced in Jpn. Pat. Appln. 
KOKAI Publication No. 8-249349, Jpn. Pat. Appln. KOKAI 
Publication No. 10-49542 and Jpn. Pat. Appln. KOKAI 
Publication No. 10-55366. 

In Jpn. Pat. Appln. KOKAI Publication 
No. 10-49542, there is proposed that a region is 
divided by using a change in color or luminance in 
images in the beginning, and the feature of each of 
the divided regions is extracted, followed by analyzing 
a topological relationship between regions and 
referring to a dictionary so that the key words for 
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retrieval is automatically added to images. 

Furthermore, in Jpn. Pat. Appln. KOKAI Publication 
No. 8-249349, Jpn. Pat. Appln. KOKAI Publication 
No. 10-55366, there is proposed an apparatus of 
5 dividing an image into regions by using a change in 

color or luminance in images, subjecting divided 
regions to integration processing to transform the 
regions into a concept of an upper layer so that a high 
degree of key word is automatically added to the image 

10 as a key word. 

These apparatuses are intended to automatically 
add key words on the basis of features of an object in 
the images , and the apparatuses use a knowledge 
dictionary such that a blue sky is disposed at an upper 

15 section of the image and has a blue color, or the like. 

Consequently, there remains a problem such that 
how a parameter value for classification should be set 
in the case where classification which is not seen in 
knowledge dictionary is desired. 

2 0 As means for coping with any circumstances, 

there is available a classification apparatus provided 
with a supervised learning. As such example, there 
is available Jpn. Pat. Appln. KOKAI Publication 
No. 11-34450. 

2 5 Hereinafter, a conventional example shown in Jpn. 

Pat. Appln. KOKAI Publication No. 11-34450 will be 
explained in detail for some time. 
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FIG. 6 is a block diagram showing one example of 
a conventional classification apparatus provided with 
a supervised learning. 

In FIG. 6, reference numeral 1001 denotes an image 
5 input and display section for inputting and displaying,^ 

for example, a wafer image. 

Furthermore, reference numeral 1002 denotes a 
defect image selection section with which an operator 
monitors a displayed image and selects a defect image. 
10 Furthermore, reference numeral 1003 denotes a 

calculation section for comparison with a reference 
image for calculating a difference from the reference 
image . 

Furthermore, reference numeral 1004 denotes a 
15 defect region extraction section for conducting a 

threshold value processing associated with the 
calculation result of the calculation section for 
comparison with the reference image to extract a defect 
image . 

2 0 Furthermore, reference numeral 1005 denotes a 

teacher signal (category signal) adding section for 
adding a category name which is a teacher signal at the 
learning step. 

Then, reference numeral 1006 denotes a feature 
25 extraction section for extracting features of the 

extracted defect region. 

Furthermore, reference numeral 10 07 denotes 



a classification parameter learning section for 
conducting learning of the classification parameter so 
that the learning data of the attached with a teacher 
signal is well classified. 

Furthermore, reference numeral 1008 denotes 
a category determination section for determining which 
of the category the defect region belongs to. 

Furthermore, reference numeral 1009 denotes a 
category name adding section for adding a determined 
category name to the defect image. 

Next, an operation thereof will be explained. 

FIGS. 7A and 7B are flowcharts shown for 
explaining an operation of a conventional 
classification apparatus provided with a supervised 
learning. 

At first, the flow of the learning step will be 
explained. 

In the beginning, at the image input and display 
section 1001, for example, a wafer image is input and 
displayed (step ST 1001). 

Next, at the defect image selection section, an 
operator selects a defect image for the training data 
(step ST1002) . 

Subsequently, at the calculation section 1003 for 
comparison with the reference image, a difference 
between the selected defect image and the reference 
image is calculated (step ST1003). 
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This calculation result is subjected to the 
threshold value processing at the defect region 
extraction section 1004 to extract the defect region 
(step ST1004) . 

5 The operator adds the category name which is a 

teacher signal to the defect region at the teacher 
signal (category name) adding section 1005. 

Next, at the feature extraction section 1006, the 
features is extracted from the extracted defect region 
10 at the feature defect section 1006. 

Subsequently, at the classification parameter 
learning section 1007, the classification parameter is 
learned so that the learning data attached with the 
teacher signal so that the learning data attached with 
15 the teacher signal is well classified (step ST1007). 

Thus, the learning step is completed. 
Next, the classification step will be explained. 
A rough flow of the learning step is approximately 
the same as the learning step, so that only different 
20 processing section will be explained. 

In the beginning, at the learning step, the defect 
image for training data is selected. However, at the 
classification step, all the classification object 
images become an object (step ST1009). 
2 5 Furthermore, naturally, it is not required to add 

the teacher signal. 

Instead, a category determination is made to 
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determine which of the category the signal belongs to 
(step ST1013). The classification step will be 
completed by adding the name of the category (step 
ST1014) . 

5 As shown above, an example of the classification 

apparatus provided with the supervised learning has 
been explained according to Jpn. Pat. Appln. KOKAI 
Publication No. 11-34450. 

By the way, Jpn. Pat. Appln. KOKAI Publication 

10 No. 11-34450 discloses a proposal on a defect 

classification apparatus in a defect inspection of 
a semiconductor wafer or the like. 

Consequently, such defect classification apparatus 
is capable of detecting the position of defects from a 

15 difference between images with defects and reference 

images. In the case where an object region such as 
general images without any reference images is 
classified, it is extremely difficult to detect only 
regions to be classified. 

2 0 However, since the device for automatically adding 

key words for image retrieval, and an automatic image 
classification apparatus are constituted in the above 
manner, so that the detection of the classification 
target region becomes a large issue. 

25 As one method of detecting the classification 

target region, there is available a method of selecting 
and discarding and integrating each region which is 



divided by referring to the knowledge dictionary. In 
the case where it is desired to make a classification 
which cannot be found in the knowledge dictionary, 
there remain a problem as to how the value of the 
classification parameter value can be set. 

A learning type classification apparatus can solve 
this problem, and can store what is learned as 
knowledge. 

A structure of the classification apparatus 
provided with a supervised learning according Jpn. Pat. 
Appln. KOKAI Publication No. 11-34450 is convenient for 
inspecting a deviation from the reference image. 
However, it is extremely difficult to clip 
classification target region at the learning phase from 
general images. 

As a consequence, work is considered to manually 
clip classification target region steadily at learning 
phase. However, such work is not preferable because a 
large quantity of labor is required. 

BRIEF SUMMARY OF THE INVENTION 

An object of the present invention is to provide a 
learning type image classification apparatus which is 
capable of clipping classification target region at the 
learning phase, or alleviating the burden of work such 
as instruction for classification target region or the 
like and a method thereof, and a recording medium on 
which the processing program is recorded. 
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In order to achieve the above object, according to 
the present invention, there is provided 

(1) a learning type image classification apparatus 
which is capable of classifying a plurality of images 

5 on a predetermined reference, the apparatus comprising: 

region clipping mode selection section which is 
capable of selecting a mode of clipping from the images 
a region out of a plurality of candidates; and 

region clipping execution section for clipping 
10 regions from the images in a mode selected with the 

selection section in the region clipping mode. 

According to the present invention, in order to 
achieve the above object, there is provided 

(2) the learning type image classification 

15 apparatus according to (1), wherein the apparatus is 

operated at least at the learning step and at the 
classification step after learning, and the region 
clipping mode selection section and the region clipping 
execution section are operated at the above learning 

20 step. 

Furthermore, according to the present invention, 
in order to achieve the above object there is provided 

(3) the learning type image classification 
apparatus according to (1), wherein the candidates of 

25 image clipping mode include a mode of automatically 

dividing the image into a plurality of regions by using 
a change in color and luminance value and clipping from 
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images a region obtained by integrating regions 
selected from a plurality of these divided regions. 

Furthermore, according to the present invention, 
in order to achieve there is provided 
5 (4) the learning type image classification 

apparatus according to (1), wherein the candidates 
of the mode of clipping regions include a mode of 
clipping the region by using a category which has 
been learned. 

10 Furthermore, according to the present invention, 

in order to achieve the above object, there is provided 

(5) the learning type image classification 
apparatus according to claim 4, wherein the mode of 
clipping the region judges the reliability of the 

15 region clipped by using the learned category. 

Furthermore, according to the present invention, 
in order to achieve the above object, there is provided 

(6) the learning type image classification 
apparatus according to ( 4 ) , wherein the mode of 

2 0 clipping the region counts the number of clipped 

regions by using the learned category, and the image 
having a region number of 1 is extracted as a learning 
image . 

Furthermore, according to the present invention, 
25 in order to achieve the above object, there is provided 

(7) the learning type image classification 
apparatus according to (4), wherein the region clipping 
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mode displays the result of the region clipping and the 
operator can have an option as to whether the result of 
the region clipping is to be adopted. 

Furthermore, according to the present invention, 
5 in order to achieve the above object, there is provided 

(8) A learning type image classification method 
which is capable of classifying a plurality of images 
on a predetermined reference, the method comprising the 
steps of: 

10 selecting a mode of clipping a region from the 

images from a plurality of candidates; and 

clipping a region from images in the selected 

mode . 

Furthermore, according to the present invention, 
15 in order to achieve the above object, there is provided 

(9) a recording medium in which a processing 
program is recorded for classifying a plurality of 
images on a predetermined reference with a computer, 
the processing program comprising: 

2 0 a first program providing a computer with a mode 

of clipping a region from images selected from a 

plurality of candidates; and 

a second program allowing the computer to clip a 

region from the images in the above clipping mode; 
2 5 wherein the first program and the second program 

constitute a learning type image classification 

program. 
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Furthermore, according to the present invention, 
in order to achieve the above object, there is provided 

(10) a learning type image classification 
apparatus provide with a learning step and a 

5 classification step after learning for classifying a 

plurality of images or automatically adding a key word 
for retrieval to images, the apparatus comprising: 

a learning step execution section for executing 
the learning step; 
10 a classification step execution section after 

learning for executing the classification step after 
learning; and 

a region clipping mode selection section provided 
with a plurality of region clipping modes, the section 
15 being capable of selecting several modes out of the 

plurality of the modes. 

Furthermore, according to the present invention, 
in order to achieve the above object, there is provided 

(11) the image classification apparatus according 
2 0 to claim 10, wherein the region clipping mode comprises 

a semi-automatic region clipping mode, a learning usage 
region clipping mode, and a interactive type region 
clipping mode. 

Furthermore, according to the present invention, 
25 in order to achieve the above object, there is provided 

(12) the learning type image classification 
apparatus according to claim 11, wherein the 



- 13 



semi-automatic region clipping mode comprises a region 
dividing section for automatically dividing the image 
into several regions by using the change in the color 
and luminance in the image; 

a region designation section which is capable of 
designating a classification target region out of the 
regions divided at the region dividing section; and 

a region integration section for integrating the 
region designated with the region designation section 
to set the region as the clipped region. 

Furthermore, according to the present invention, 
in order to achieve the above object, there is provided 

(13) the learning type image classification 
apparatus according to (11), wherein the learning usage 
region clipping mode comprises: 

a section for defining a region relationship 
between a category which has been learned and a 
category which is being learned; and 

a region clipping execution section by the 
category which has been learned for executing the 
region clipping by the category which has been learned. 

Furthermore, according to the present invention, 
in order to achieve the above object, there is provided 

(14) the learning type image classification 
apparatus according to claim 11, wherein the 
interactive type region clipping mode comprises: 

a section for defining a region relationship 
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between categories for defining a region relationship 

between the category which has been learned and the 

category which is being learned; 

a region clipping execution section by the 
5 category which has been learned for executing region 

clipping by the category which has been learned; 

a region clipping result display section; and 

a judging section for judging the adoption of the 

region clipping. 
10 Furthermore, according to the present invention, 

in order to achieve the above object, there is provided 

(15) the image classification apparatus according 
to claim 11, wherein, the learning usage region 
clipping mode further comprises a region clipping 

15 reliability judging section for judging the reliability 

of a region clipping wherein the system judges the 
reliability of the region clipped by the category which 
has been learned. 

Furthermore, according to the present invention, 

20 in order to achieve the above object, there is provided 

(16) the image classification apparatus according 
to claim 11, wherein the learning usage clipping mode 
further comprises: 

a region number counting section for counting the 
2 5 number of regions of the result of the region clipped 

by the category which has been learned; and 

a learning image candidate selection section in 
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which an image in which the number of regions is 
counted 1 is set as a candidate of the learning image. 

Additional objects and advantages of the invention 
will be set forth in the description which follows, and 
in part will be obvious from the description, or may- 
be learned by practice of the invention. The objects 
and advantages of the invention may be realized and 
obtained by means of the instrumentalities and combina- 
tions particularly pointed out hereinafter. 
BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING 

The accompanying drawings, which are incorporated 
in and constitute a part of the specification, illust- 
rate presently embodiments of the invention, and 
together with the general description given above and 
the detailed description of the embodiments given 
below, serve to explain the principles of the 
invention. 

FIG. 1 is a block diagram showing a structure of a 
learning type image classification apparatus according 
to a first embodiment of the present invention. 

FIGS. 2A and 2B are flowcharts shown for 
explaining an operation of the learning type image 
classification apparatus according to the first 
embodiment of the present invention. 

FIGS. 3A, 3B and 3C are views shown for explaining 
as types of region clipping modes of FIGS. 2A and 2B 
three modes; a semi-automatic clipping mode, a learning 
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usage region clipping mode, and a interactive type 
region clipping mode. 

FIG. 4 is a block diagram showing a structure of a 
learning usage region clipping mode out of region 
5 clipping modes in the learning type image 

classification apparatus according to a second 
embodiment of the present invention. 

FIG. 5 is a block diagram showing a structure of a 
learning usage region clipping mode out of region 
10 clipping modes in the learning type image 

classification apparatus according to a third 
embodiment of the present invention. 

FIG. 6 is a block diagram showing one example of 
the conventional image classification apparatus 
15 provided with a supervised learning. 

FIGS. 7A and 7B are flowcharts shown for 
explaining an operation of the conventional image 
classification apparatus provided with a supervised 
learning. 

2 0 DETAILED DESCRIPTION OF THE INVENTION 

Embodiments of the present invention will be 
explained by referring to the drawings hereinbelow. 
(First Embodiment) 

FIG. 1 is a block diagram showing a structure of a 
2 5 first embodiment including a learning type image 

classification apparatus, a learning type image 
classification method and a recording medium on which a 
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learning type image classification program is recorded. 

In FIG. If reference numeral 101 denotes an image 
input and display section. 

Furthermore, reference numeral 102 denotes a 
5 classification category content receiving section for 

allowing the system to receive which type of 
classification category is made. 

Furthermore, reference numeral 103 denotes a 
region clipping selection section for selecting the 
10 mode in which the user or the system conducts the 

clipping processing of the region corresponding to the 
category in the image. 

Furthermore, reference numeral 104 denotes a 
learning type image selection section used at the time 
15 of category learning. 

Furthermore, reference numeral 105 denotes a 
region clipping execution section for clipping a region 
corresponding to the category from images in any of the 
modes selected at the region clipping mode selection 
20 section 103. 

Furthermore, reference numeral 106 denotes a 
teacher name (category name) adding section for giving 
in set a category name which becomes a label of 
classification category. 
25 Furthermore, reference numeral 10 7 denotes a 

feature extraction section for extracting features 
representing the feature of the clipped region which is 
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extracted. 

Furthermore, reference numeral 108 denotes 
a learning section of a classification parameter for 
adjusting parameters in the system so that the set of 
5 feature data can be classified on the basis of the 

teacher signal. 

Furthermore, reference numeral 109 denotes a 
classification object image input section for inputting 
images which are desired to be classified after 
10 learning. 

Furthermore, reference numeral 110 denotes a 
region dividing section for dividing images input after 
learning into several regions. 

Furthermore, reference numeral 111 denotes a 
15 category determination section for determining the 

category of images input after learning. 

Furthermore, reference numeral 112 denotes a 
category name adding section for adding a determined 
category name to an image or a region. 
2 0 Furthermore, reference numeral 113 denotes a 

recording medium in which there are recorded a first 
program for providing with a computer a mode of 
clipping a region from selected images from a plurality 
of candidates and a second program for clipping a 
25 region from among the image in the clipping mode to the 

above computer. 

Next, an operation of the apparatus will be 
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explained. 

FIGS. 2A and 2B are flowcharts shown for 
explaining a first embodiment including a learning type 
image classification device and a learning type image 
5 classification method and a recording medium in which 

the learning type image classification method is 
recorded. 

In the beginning, the learning step will be 
explained. 

10 At the outset, at the image input and display 

section 101, the image is input to this device or the 
system and the image is displayed on the screen ( step 
STIOI) . 

Next, the user determines how the user desires to 
15 classify the images while watching the image group 

displayed on the screen and notifies the content to the 
system at the classification category content receiving 
section 102 (step ST102). 

Specifically, the system assumes a certain 
20 degree of classification need, or allows the user 

to select the need from among the options such as, 
for example, it is desired that human face images 
are classified apart from other images, or it 
is desired that face images on an individual level 
2 5 are dealt with. 

Next, the region clipping mode selection section 
103 makes a selection as to in what mode a region 
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corresponding to the category is clipped from the image 
(step ST103) . 

As types of region clipping modes ,^ there are 
available, for example, a semi-automatic region 
5 clipping mode, a learning usage region clipping mode, 

a interactive type clipping mode and the like. 

Hereinafter the three modes will be explained by 
using three FIGS. 3A, 3B and 3C. 

At the outset, there will be explained 
10 a semi-automatic region clipping mode shown in FIG. 3A. 

In the beginning, the system automatically divides 
input image into regions by making use of the change in 
color and in luminance values . 

Subsequently, at the region instruction section 
15 A2, some of the regions divided are clicked by men. 

Consequently, at the region integration section 
3A, the system integrates and processes these regions 
and adopts the integrated region as the result of the 
clipping of the regions. 
2 0 Next, there will be explained a mode of learning 

usage clipping mode shown in FIG. 3B. 

This is a one touch style region clipping mode 
which does not allow users to be conscious of the 
clipping of regions. There are several limitations in 
25 the usage of this mode. 

That is, this is a method of attributing the 
clipping of the regions of categories to be classified 
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from now on to the detection of regions of the category 
which has been learned. 

For this purpose, in the section Bl of defining 
a region relationship between categories, the 
5 relationship between regions with the category to be 

learned are given to, for example, to the dictionary as 
knowledge . 

Then, in the case where the relationship between 
regions agrees with each other, it is possible to 

10 attribute the clipping of the category region to be 

learned to the clipping of the region of the category 
which has learned. 

For example, when learning is conducted which 
allows differentiation of human face images and other 

15 images, human face images are clipped by using 

parameters which have been learned to add category 
names of the individual's names to the clipped regions 
at the region clipping execution section B2 by the 
category which has been learned in the case where 

2 0 classification is made which allows differentiation of 

images on the individual level. 

When this mode is used, the user is not required 
to consciously conduct troublesome region clipping 
processing only by collecting images for learning and 

2 5 giving a teacher signal. 

Thirdly, there will be explained a interactive 
type region clipping mode shown in FIG. 3C. 
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This is a method for displaying the result of the 
clipping of the face region explained in the learning 
usage region clipping mode and selecting and discarding 
images used in learning by the user ' s confirmation 
5 of the region detection ratio in the clipping result 

at the region clipping result adoption judgment 
section C4 . 

Then, returning to FIGS. 2A and 2B, after the 
region clipping mode is selected, the learning image 

10 selection section 104 allows the user to select a 

learning image (step ST104) and the region clipping 
execution section 105 clips the classification target 
region from among the images in the style of respective 
modes (step ST105). 

15 Next, in the teacher signal (category name) adding 

section 106, the classification category names such as 
faces and forehead or the like are given in sets to the 
image or the clipped region in set (step ST106). 
Next, in the feature extraction section 10 7 

2 0 extracts the feature such as color, shape texture or 

the like from the designated region (step ST107). 

After that, at the classification parameter 
learning section 108, a feature vector is prepared from 
the extracted feature and a classification parameter is 

2 5 learned and determined so that the feature parameter 

can be classified in accordance with the teacher signal 
(step ST108) . 
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At this time, processing such as overlapping the 
feature in accordance with an effective degree for 
classification (for example, Jpn. Pat, Appln. KOKAI 
Publication No. 9-101970) may be used. 
5 Incidentally, as a result of the learning of the 

classification parameter, for example, at a kohonen 
type neural net, the neuron element corresponds to a 
representative vector in the vector quantization so 
that the neuron element is set in a state in which a 

10 label representing the classification category name is 

added to this representative vector. 

By the way, the learning is thus completed, and 
now the process proceeds to the classification so that 
the process now proceeds to the classification step. 

15 In the beginning, the image input and display 

section inputs or displays a group of images (step 
ST109). 

Next, at the classification target image input 
section 109, the image which is desired to be 
20 classified is input (step STllO). 

After that, at the region dividing section 110, 
the region is divided by using the change in color and 
in luminance value (step STlll). 

Furthermore, at the feature extraction section 
25 107, the feature such as color, shape, texture and the 

like is extracted from each of the divided regions 
(step ST112) . 
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Next, at the classification parameter learning 
section 108, the feature vector is generated from the 
set of feature obtained in each region 108. 

Next, the feature vector of each region is 
5 classified by using a classification parameter which 

has been learned and the category determination section 
111 determines to which category the region belongs 
(step ST113) . 

Lastly, at the category name adding section 112, 
10 the determined category name is added to the region or 

the image (step ST114). 

As explained above, processing flow at the 
learning step and the classification step after 
learning has been explained. However, the learning 
15 type image classification program which is executed at 

each section is recorded on the recording medium 113. 
(Second Embodiment) 

FIG. 4 is a block diagram showing a structure of a 
learning usage region clipping mode out of the region 
2 0 clipping mode in the learning type image classification 

apparatus according to a second embodiment. 

The structure of the second embodiment is the 
approximately similar to the structure of the first 
embodiment . 

25 A main difference of the second embodiment from 

the first embodiment is that a reliability judging 
section B203 for judging the reliability of the 
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clipping of the region for allowing the system to judge 
the reliability of the region clipped by the category 
which has been learned is provided in addition to the 
section B201 for defining the region relationship 
5 between categories and the region clipping execution 

section B2 02 by the category which has been learned. 

For example, the reliability is represented in 
numeric values, so that the teacher data of an image 
having a high numeric value is used. 

10 Otherwise, when the numeric value representing 

the reliability in the face detection in an image 
group selected by the user is low, the process 
proceeds to the semi-automatic clipping mode for 
clipping the region corresponding to the category in 

15 the image. 

Consequently, in order to alleviate the work 
burden of the region clipping, it is possible to avoid 
a situation in which the precision of the region 
clipping has been deteriorated. 

20 (Third Embodiment) 

FIG. 5 is a block diagram showing a structure of 
the learning usage region clipping mode out of the 
region clipping in the learning type image 
classification apparatus according to the third 

2 5 embod ime n t . 

The structure of the third embodiment is 
approximately similar to the structure of the first 
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embodiment . 

A main difference of the third embodiment from the 
first embodiment is that a region number counting 
section B303 for counting the number of regions of the 
5 region clipping result by the category which has been 

learned and a learning image candidate selection 
section are provided in addition to the section B301 
for defining a region relationship between categories 
and a region clipping execution section B302 by the 
10 category which has been learned. 

For example, classification is made on the 
individual's face level, learning is conducted for 
classifying human face images from other scenery 
images in advance, the individual's face image 
15 region is clipped by using the clipping of this face 

region. In the case where a plurality of faces are 
photographed on one image, it is required for the 
user to designate which face region is about to be 
classified. 

2 0 In the third embodiment, only in the case where 

the number of human face region is one, the face region 
is set as a candidate of the learning image. 

As has been explained above, according to each of 
the embodiment of the present invention, there is 

2 5 provided a region clipping mode selection section, so 

that there is an advantage in that the burden of the 
clipping processing of the object region by man can be 
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largely reduced at the learning step of the image 
classification. 

In particular, according to the invention claimed 
in claim 10, a region clipping mode selection section 
5 is provided, so that the burden of the region clipping 

by the user is light, and, the region clipping mode 
having a favorable clipping performance can be selected 
with the result that there is an advantage in that the 
burden of the region clipping at the learning phase by 
10 the user without deteriorating the classification 

performance . 

Furthermore, according to the present invention 
according to claim 15 described later, there is 
provided a reliability judging section for judging the 

15 reliability of the region clipping. 

When the numeric value representing the 
reliability is low in the face detection in an image 
group selected by the user is low, the process proceeds 
to the semi-automatic clipping mode for clipping the 

20 region in the region corresponding to the category in 

the image . 

Consequently, in order to alleviate the work 
burden of the region clipping, it is possible to avoid 
a situation in which the precision of the region 
25 clipping has been deteriorated. 

Thus, the present invention has an advantage in 
that the classification precision can be improved. 
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Furthermore, according to the present invention 
claimed in claim 16 described later, there are provided 
a region number counting section, a teacher signal 
image candidate selection section with the result that 
5 there is an advantage in that man can omit the work of 

selecting and designating the object region from a 
plurality of regions by selecting only images having a 
region number 1 as a candidate of teacher images so 
that time wait problem up to the selection and 
10 designation can be avoided. 

Incidentally, in each of the above embodiments, 
focus is placed on the clipping of regions at the 
learning step. 

However, the present invention is not necessarily 
15 limited only to the clipping regions at the learning 

step. 

That is, the present invention is particularly 
suitable to the learning step, but the present 
invention is not limited thereto. 

20 Consequently, as has been explained above, 

according to the present invention, there can be 
provided a learning type image classification apparatus 
which is capable of clipping classification target 
regions at the learning phase and which is capable of 

25 alleviating manual work burden such as instruction of 

classification object regions, a method thereof and a 
recording medium 113 on which the processing program is 
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recorded. 

Additional advantages and modifications will 
readily occur to those skilled in the art. Therefore, 
the invention in its broader aspects is not limited to 
5 the specific details and representative embodiments 

shown and described herein. Accordingly, various 
modifications may be made without departing from the 
spirit or scope of the general inventive concept as 
defined by the appended claims and their equivalents. 



